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Tills docunent is a suirxiary of a number of ^;T,'La].l inlicvjce 
invGStifjations perforuicd by *-.]ig CorapuLcr Based Projcict for the 
Evaluation of .>edia for the Handicc>ppcd during:; the per .od September 
1969 t]iroii[^h July 1973 r I-'umcrous persons v/ero iuvolv^^d in t]\e 
collection of data.^ discus5i!lou of findinp;s and iiaplciineutntlon of 
decisions based on the studies, and these are noted in the ircfercnces. 

The research and evaluation effort of the Projjcct durin;;; the 
four years covered by tlie report has developed froin the follov/ing 
nssuniptlons : 

a) A number of questions arc generated :::s one proceed-^ to 
v7ork V7ith data from educational activities - 

b) Questions relctinj^ up specific project decisions inu^it be. 
investigated to give the decision a basis for Vc'.lidity- 

c) The investi[i;ation of all questions suggested or generated 
is inipos^jible, 

d) Staff nei^ibers have r>pecific interests that .can be met. 

e) A frarae\;orlc of general J uvesti^^^ative parcinioters should 
bo npeci f iod . 
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Research was needed as Project efforts began. General questions 
arose concerning identification of the educable handicapped child, 
his capabilities and liraita^ ions , and his responses to vaediated in- 
struction. To keep the research effort frcn fragmenting in different 
directions J five major topics were identified to guide investigators; 
the effects of pretestings best response i:;odes, the effects of captioned 
films > student characteristics related to raedia use, and the miscellany 
of data needed to support operative decisions. 

Apparent differences in the reporting cf findings in the section 
suinnaries and those appearing in the original report are recognised 
as being the result of this author's having access to all naterial 
at one time and the inevitable differences in conceptualization of 
two or more individuals. 

This suimnary is presented in an effort to disseminate the findings 
of the Project and to expand the knov?ledge about effects of mediation 
on handicapped children and measurenient problems associated x^/ith 
determining a material's ef f octivenef^s. Publication here does not 
preclude the I'iter inclusion of part or all of the documo.nt in educa- 
tional journals or other professional publicatiors. 

Jack Ho Bond 

Coniputer Based Project 

Syracuse, Hew ^York 
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Chapter I 
Introduction 



Since initiation ia iii:nni-r>ii<3 CiC*:iv5/cio^ undorl:a?:i-;n nt C'vr.ynitcr 

Based Project in accordance with contract obli[!ationii with '[cdia Sex'vicos 
and Captioned Filras have been in the nature of qiinni-expnrinental research 
and/or inve3tip;ative ntudies. This Report is a co^v.posite of such investi- 
p,ations publij^lied to f^hare with otlicrs interested in evaluations, particu- 
3.arly media evaluation > and special education the insi^^hts [gained and the 
implications Oj: v/ork done at CBP. Though riore que.stions x-;cre uncovered 
throu^li the invo^jitij^ati-^^e efforts than ansv-ered, th.e sur^ir^aries contained 
lierein v;ill aid other investigators in avoiding i^any of tl;e problems 
encountered p foraulatin^; nore appropriate questiouS; and extendinf^ the 
efforts begun. 

The CoTTiputer Baced Project for the Evaluation of "ledia foi- the 
Uandicapped (CjJP) vas funded by J'edia Services and Captioned Films i'S/CV), 
h\iV2ix\\ of Education for the llandicanped (BEII) 5 in the suimier of 1969; 

(a) To demonstrate cm effective and efficient systen of evaluating 
aj:d field tes:;tin,<> educational materials v?ith a variety of 
handicapped children. 

(b) To demonstraiie an effective technique for educating handicav;ped 
children v/ith a nsjor eiviphasis upon n^-'yj.rniij?. support of tlic 
learning process through ir.c-idia, tutorin;^, team taacijinp;^ and 

a sysLOii!s approach to education. (CB? Proposal, 1959) o 
Cijp h.as applied the sysCOir.s ni-proacii to the l(».stinrr cx,c\ evaTuating of 
instructional "t.:av:erialci for haidicapped childrn and is previdinp, 
iupDrvnation and services for the Imprnvcricvit of let::rnjn;' of liai^d:^ capped 
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children throu^jh the apj^ropriate use of instructional technology. The 
CBP staff has developed an evaluation Tccdel based on student responses 
to instructional materials and studied the related issues of the inter- 
action faetueen student characteristics and media characteristics. In 
addition.^ the CBP staff has been involved in teacher-training and curriculum 
implications of increased media use and dissemination of evaluation results. 

During its last year, CBP Xvill coiaplete the evaluation of the films 
in the HS/CF depositories. This evaluation involves using captioned 
filniSj designed and provid-jd for the deaf 3 with other educationally handi- 
capped populations. C33P V7ill also use the evaluation iiiodel to determine 
the effectiveness of media other than captioned films and to expand the 
evaluation of the HS/CF depository holdings V7itb deaf populations, 

Tl'ie Evaluation llodel 
The CBP evaluation model is the process tVirough which CBP evaluates 
media; particularly captioned media. The model encompasses several 

nocnfiipt-i ons « , 

(a) Children can provide vital - information for making decisions about 
the. effectiveness of media. 

(b) Objectives of materials should be specified. ^ 

(c) Criterion items can be developed and provided for practitioners 
to measure the expected outcomes. 

(d) Characteristics and outcome parameters may be assigned so that 
comparisons beurceu media can be made. 

Bond (1972) has described the evaluation process in considerable detail 
It is .summarised in the flovr chart and discussion below (adapted from 
Miller, 1973). 
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(3) Develop Scilectiovi | ^/.p-.roprir tc^n -.s^/" 

Criteria i 



\ 



/{) e t c V n in e S p o c J . f i 
(C) Objectivcfi for 
\ each Piece oi: 
\ n-Ji:i _ 



} / Develop 
i (^(7)- Criueria 
! \ IteiT.s 



o 



\ 



for Teytiiif; 



Select Sa:ti])le 
^^^of Studeiil s 




(14) Decisiond-;) 



FIGURE 1-1 Evaluation 'iodt^. Tlo^i Chart 
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Discussion 



Assess Curriculun/ 
Learner i'eeuG 



2. Develop Selection 
Critci'ia 



3. Iledia from Producer 



Tho prOT.ise is that nec'ia should fit 
the nc.c.ds of t!ie j.i:t'irnGr. Ideally j 
iTt'^dia .".hould ho. pr-vpared to raeet the 
particular neeor^ of each learner or 
group of loornors. 

The selection criteria are stivjiulated 
by the needs asGC?:ifjcd in 1. These 
criteria allow the evaluator t:o narrov; 
the search to relevant material • 

IJ/or v;/o objectives nvA test iterus* 



Evaluation Proaese- 



4. Ilevlev! Iraterlal 



5. Decision 



Accept 



Reject 



Final Peeper t 



6. Determine Specific 
Objectives for each 
piece of Iledia 



Develop Criteria Items 



This is the initial screening step. 
Criteria developed at Step 2 are 
applied here. 

Iledia that does not pass the initial 
review are rejected, and a report 
specifying the deficiencies is vjrltten. 

If TBaterial passes the initial revievj^ 
it is then scheduled for the rest of 
the evaluation procedure • 

General objectives are determined by 
curric;ilum/ learner needs. Step 1. 
Specific objectives are extracted from 
each piece of media depending on its 
content, format, etc* 

A set of criteria-referenced test items is 
developed for eacli piece of Tiedia, 
based on the objectives extracted in 
Step &• Care should be taken on the 
formulation of test instrunients ♦ 



Prepare I'laterials for Testing The test items are photographed as 

slides for presentation to the 
students. 



Select Sample of Students 



This step may be done at any time 
after the appropriate upper level 
of the film is established. 



10, PreseMit Materials to Students Requires normal effort in scheduling s 

etc. MB,. Shoxm to highest appropri-- 
ate group. 



12. ;Mio\?a.n£<s to Otlior Groupr 



13 » Fin<al Report 



.'^ci'^.lnli^tcr Cr i l' or ia- Po.r cr »^ncod Tcs t . 
etc. Cle:\r Gtr.tcn-p/nts oL tlie factors 

to bo a:Unlyc;od I'lH ? t DG mitG . P.QCOn- 

moiiclrttions ror objectives j itcns> other 
grovips and the frlr'rJ. report ^ro urcd qg 
inpuLr* rlnto Stoip 65 7? , <.nd 13 ra- 
sp n:: lively . 

If incoTTipleto decisions rcr'n iin the 
process recycled into Ste^- 9 for more 
or different j;^,roups. 

This report nay be dcscr:' nl:ive/ 
forraative or iud[^;'.aental/L--U:TUi;ative 



14, Decision (g) 



Reject 

Purchase or Rent 
Shovj to Ulion 



Trior to ruikinp '^'decirions' , decisi.ons 
on decirions nu.it he nadc. Belov^ are 
three possible deci?ica.'^: . 

1.. 1'Jhat decisleuG are c^^^r^ be made? 

2. ITuo is s^Jinj: to make '-he decision? 

3. \Jhat criteria are to be useid 
for the docisiox^? 



Pro;}cot Feacarc^h and disser:lruCition 
Initially inves ti^ativo activities conducted for the purpose of 
mcctinf^ contract oblic.^^-tionG v;ere recorded in-house^ fulfillin[r the need 
to substantiate procedure aecicions v:ith data. In 1970 the site visit tcan 
sunocstcd that these investiriatious be expanded and reported to llir^ited 
audiences ruich as Project consultants and rs/CF (Site Visit 'leporti 1970). 
As the evalnntdon procedure f;;ot undertrayj a nutTibcr of related interests 
V7ere generated* and recommendaticns T;ore nade to make Project results 
available to the larc^^-r f ield-^def ined audience (Site Visit P.cportj 1972). 
A disseiiunation objective v/an devc3opcd during the fourth year of Project 
operation and a{],recrt^eiit made to produce a sut:.:?.ary of Project inve:*ti[^ations . 
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TUg t':uords i:ub*-'. nntiatinf; t:he Projcct'c i-cr^oarch- type activities 

V 

hove cimilnrly developed frora inforriial invest i f-.a Live reports to formal 

pvoceduruo for nroblr" cpeclf ic:»tlon , data forr^ and rc:>'^^- tir.f^ . Early 

studies conf^isted merely of a file folder \7ith nv.j data and notcfi, 

tosether v;ith a concluding stateiaent of findiniv^ precented to the staff. 

During the i^econd and third year of Project or»erations invcsitijative 

reports included data coripilation. In 1973 re{:oarch-typc activities 

vjere formali::ed v/ith ^vOecific assigning of staff., sample, and instruments > 

as v;ere tho research report format and procedures. 

The smvimary of -oports dravjs frora both the inforrual investisatioiis 

and formal research^ the purpose beinj3 to make available to those 

interested tlio results of Project activities. 
I 

Ter-ms and Ahbreviatio'r.s 
Because of the frequency of a number of titles and the special 
r:eaning that these and other terms have specific to the Project , titles 
are specified and terras defined as follows: 

Clironological Age (C'\) : /^ge of a student determined by subtracting his 
date of bi:^th from Deccrabcr 1 of t'ue current school year. 
EdiiC(jbte Urmtally Handicapped (El ill) Children vho have scored betvjcen 
50 and 75 on a standard individual I.Q. test and have been jud{;;cd to 
benefit from a special class placement. The Project v?orhs vjith 200 - 
300 of about 900 available subjects^ 

Expevimento^'* 0'-) ^ Staff member r'anii>ulatin[? or observiun the situatJon, 
material, or child in a way that has been defined and outlined to obtnia 
specific irLformation particularly for a study or jVnv.3.st j{^;ation. 



ERLC 



- 6 - 



IntelKijencc QiiOiicni: (IQ) : A r^tio dd'crrainod fr-r. the rnrntal ci^.a 
and clironolo;^ical o;;;o c2 a subject a^ter hejviri^. hc'::n ad;>inistercd n 
standard IQ tent. Tbo two indiv idvial tcni:i; uith project -subjects 

are the Stanford Binct and the Uc::ler IntellinencG Scale for Childrcin 
(Wise). Both tests give a full scale (FS) ratio and the WISC also 
conpites a perforn^anne (PIQ) and a verbal (VIQ) r;ub ratio. All thrae 
values are usrjd if available, 

iJental Ago ('JA) ; Determined froia a tabled value obtained by a t\;o- 
variable matriic consisting of CA nad the score received on the r-tan- 
dard IQ tests of Stanf ord-^Jiinet or Uexler Intelligence Scale for 
Children (UISC) . 

Operators or System Operators i Referrinij to tlie staff person uho 
controls the recording of responses usin^ tlie S]\3 eystein. 
Participant Observation (PO) : A research approach in V7hich the n?ajor 
activity is clmracteri?:ed by a prolonged period of contact with sub- 
jects in the place in uhich they noTT^unlly spend their tivne. During 
the encounters,, data^ in the forra of field notes, are unobtrusively 
and systematically collected o 

Student or Child (S) ; Child responding to a prescribed treatment or 
participating in a study. 

Student neoponoe System (S?.S) An automated system consisting of devices 
to collect student responses from pushing a button. In this docunientj 
SRS refers specifically to the Genr-ral Electric S^.S~]OnO installed in 
the Project facility. 
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Dijvaouoc y<^iolaotio nsliobilitation (SSR) Cnildren uho have emotional 
probleips of r.uch riiaj^-nivucle that they cannot be ]5lciccd in tlie rc^^'.ular 
clasfiroohu They are i>lcca:l in z'a.zII r.roupi: (r^nximun of 10) and ar^cic-'^Gd 
to indivional therapy sessions v;ii:h a psychiatric social vjorl-er and a 
special teacher. An SSR child is reassigned to the rep^ular classroom 
v/hcn ho, . '--a control his b'.^]iavior and erotions, 
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Chapter II 

Investigations Concerning Evaluation Procedures: Desiga and Technique 

The Project had available a population of special education 
children. and their teachers and a library of captioned materials to 
develop the evaluation process. Although the conducting of research 
studies vjas not the priinary purpose of the Projects the necessity of 
such efforts in developing and validating evaluation procedure \7as 
apparent to the staff. Of special concern V7ere the designs used for 
evaluation and issues of control and the techniques of data collection. 

Design 

Early data collection efforts vjere conceptu.ilized as being of 
Soloiiion Four-Group Design to verify the base line responses of students 
to a filmstrip or film. The lack of reference points for base line 
responses suggested the pretest-treatment-posttest tr.odel to obtain both 
the base line bef ore-treatinent response for ElIH children and the changes 
follov7ing treatir.ent. 



Pretest Only 




Pretest 


No Treatment 


Posttest 


Pretest 


Treatment 


Posttest 




Treatment 


Posttest 




Posttest 



Fijjure 2-1.. So3.oinon Four-Group Design 
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TliG Solomon Four provided for varif ica'cJ.on of these * asponses and the 
level of the responses in the non-treattieut setti.ng^ It v/as recoi^nised, 
hovrever, that th:i expenditure of otaff and subjecl. time for ricetiiTg ti o 
requirements of such a design would require rei^ources greater than 
those available. Furthernore , because of the chort time lapse of treat- 
ment {suc.nlng an eight to 20 minutes film) and the fact that some groups 
would be re?.;.ondir.;; to items without treatir.ent> it was rationalised 
that the inforio.ati . v obtained from the pretest only and posttest only 
cell VJOuld be pravidod e!>Gentlally by the prot-c-st -no t-Tontment i/oai t'ont 
group (Pru^:ek, 1970) and that this information could be obtained by mix- 
ing items. Thus the design .evolved as a simple treatment control v/here 
two j^roups ixxe. oh own different filr.iF*. Tlie gain comparison for Film A 
is made between Items A (Group II) and Posttest A (Group I) and likewise 
for Film B. 

Group I Treatment Film A Posttest A ■<:- Item B 
Group li Treatment Film B Posttest B Item ii 

Figure 2-2: Evolved Design 
Early comparisons of the test scores indicated that no differences 
were measured in the pretest-no treatment--pos ttest groups; therefore, 
this section of the design xeas dropped. Comparisons of the posttest 
scores of the pretest-posttest groups revealed a depressing ef.'ect of 
the pretest of about 10% (Bond, 1970) iv^hich would result in a more con- 
ooLvnf ivo eotiimate of the learning. This was thought to be desirable 
and the treatmont-pus^.f of..*- ciMjai-lon v/ns dropped. The simple pretest- 
treatment-post test model was ^ido.pted for data collection. 
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Pruzek (1971) sus^iiested that the contardnation effects of pretest on 
t:h3 same group as the posttest could be overcone by usin^, independent 
tliough equal groups for tlie pretest by mixinr^ itciTis fron a f lira not 
seen into the instrument used as the posttest for a filin seen. This 
x/ould provide for only one testing situation with, a class and the 
shov;iz3C of a filra to every class. Studies have indicated no change 
in the level of response obtained from a pretest or from the items 
mixed i'^to another test (Bond, 1973), 
Contvot iQCiueo 

Txjo schools of thought concerning control groups \;cre prevalent 

\-;ithin the consultant services available to the Project for comparisons 

in experiraental studies: that the control group is explicitly defined, 

identified^ and participates by providinf^ data as described in the 

original planj or that control is luaintained ^statistically snd the 

control group identified as it racets the criteria or participates by 

administrative error or change. For exaraplej. one could study the 

effects of saTp.ple size by i^ecifically desin;nin[> data collection for a 

siiiall group-, larger group, and still larger group and collecting such 

data.; or one rould obtain a data base v:here a number of responses are 

avai]able and randotnly dravj larger and larger samples, recomputing 

the values of the sariple statistics. The latter concept has been 

applied in ir.ost Project r.tudies. 

To verify ai:iaumptions about sapaple size and to check differences 

I. 

amonf^' population subp.roupS; specific studies UGrL-*. initiated. Under 
Pond's assumption that data from one child is better than none, the 
Project proceeded to davelon the evaluation process usinr'. very small 



independent: sarriples of 3 to 3.0 children. Send (1972) invest i.^ated 
the item rc^sponse:? of 70 children to the filnstrip Our Hands ' by 
drawin;^, savaples of 5 , 25 j nnd !30 frcrj t'.\G total pon-jIofJou (9'') and 
concluded that er>tiiuat?.s based on canples of 5 were unchcin?;ed by 
estj,mateo. based on lai^^er sariDles. 

Another issue of concern ^mg the niimber of itejns uned to mea^^ure 
a Taediur.i'' « effectiveness. Qbscrvatlon indicated that the quality of 
items produced by v;riterc; dirulnished after about 15 items had been 
created for a ^i'^'cn f ilm/f ilvi^strip . Thus from a production vie\-?polnty 
a 10 to 15 item instrument seemed desirable. Kiesau and Bond (1270) 
investigated the quantitative isr.ue of question-set size, the number of 
itens presented in a single sitting, V7ith junior hirj;h level R-'H children. 
They adninl:U:.ered a ::;et of 25 items and observed that students be<;:^an 
exhibitinjp". deviant behavior after respo?iding to 15 question s^ with the 
number of deviations and number of students involved increasing \.ath the 
number of items only 2 of 12 children responded to the 25th iten, 
Inforionl observations made by operators and teachers in the S'lS system 
establisliod 10 items as a ipaxiniun for any one presentation vrith pririary 
EUI aiid SS'^ children, j^oy and Barcilcoushi (1073) recoiiiinend 6 itein tests 
v?ith several subtests ^nade from the item set as preferable for estiiaates 
of means and standard deviations. In line T;ith the findings, the Project 
produces an item set of 10 or more items and adainisters, after filra presen- 
tation, a combined set comprising 'about 5 items from the. film seen and 5 
from a film not seen. Thus one c>^-*^^P provides baseline data on the film 
not seen and achicvenen^ data on the film seen. 
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Perceived differences aTnong popiO.ation subp.rounsj Kill and SSll, 
provided a third control issue and were e:<a?ained. vio signif icr.it 
difference on test: responfies V7as found bet^en educabla mentally 
handicapped (Kill) and emotionally dictrurbed (SS?0 populations. One 
croup of children designated E^l but e::hibitin;> considerable emotional 
instability:, ho^/ever, va<3 found significantly different in its responses 
fron both EH.! and SSR populations (Levjis, 1971). 

The question of the effects on " transporting, children to the 
evaluation facility \7as investigated durinj^ the second and third years 
of operation, vjhen the Project expanded its student population from 
125 to 300* SRS observers quar;tioned. the reliability of responses 
of transported children as corapared \rith non- transported children; 
thooe transported to the facility appeared to be more active. Bond 
and I-Jinchell (1973) found that althouf^h transported children required 
more tin\e to cinsvjor a given set of questions, there \rere essentially 
no differences in the scores obtained. 

Data Collection Techniques 
Student Response System 

Tlie GE SRS-10(10C Student Response Kystem (SRS) provided a taethod 
of monitoring; student performance in either the group or the individual 
mode. The system is coinposed of five major assemblies.* fifteen student 
stations, instruct:or 's control panels class display, interface unit, 
and output juncticai box* Each assembly is discussed in detail in a 
Project report (Q'iP,, 1970). It is appro])riato. hero to sunimaris:e by 
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saying tluit the S^.S Is one of several inethods of data collecticn used 
by the Project. For filn^i evaluation, the SRS group inode has been used 
extensively, providing input for 75% of the cognitive response base 
over a three, year period. 

The individual iriodo has been used for films trip evaluation. Four 
carrels v;cre modified and equipped with a rear projection screen, student 
GtatioUj, head set., and individual coTif irmation unit. During presentation 
of a fllmstrip or slide sequence, visual frames are synchronized V7ith the 
audio tape using the CE AVPv Digital Tape ?vGcorder for audio and slide change 
pulses and controlled by the student performing in the carrel. The 
student's responses arci nonitored by the SRS and recorded on a punched 
paper tape that is fed directly to the computer for analysis. Problems 
relating to the operation of the carrels have been reported by Plotnick (1973). 
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Fi<^,ure- 2-3' Indi^^idual Carrol Set-up 
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Item Lrmgth 

AttGuipts to obLain inf orinatioii froni children revealed a ninnbcr of 
problc'ir.s. Traditional tcstin;^ proccdi.ireG subGuirdng t;o11 rlovelonod verba] 
skills on the part of respoiidonts \jere found inadequate for E-iH children. 
Thus efforts were made to refine typo.G of verbal items used in SRS proced- 
ures and to develop attention profile and xntervicv; procedures- 

c 

Guidelines for items used in the pretests and posttcsts of media 
evaluation were established as a result of both consultant recoTmnendations 
snd investigations V7ith t'ne Project. Such [r,uidellnes apply to question- 
stem length;, nuiTiber of ansv7sr choices; ync\7er*-choice length, and level of 
vocabulary. TT.eeker (1971) suggested that EIII children could reneniber a 
laaximura of four choices j if such choices are one \7ord in len-^th and dra™ 
from familiar concepts. She further sar;3ested tliat itens be positively 
statedr Gronoski (1571)5 investigating the relationship between correct 
responses to questions and the length of ansv;er choices j found a greater 
number of randon choices and vjrong. ansvTers to long items as compared with 
short items. 

Interviews 

Use of observation and verbal interview to substantiate cognitive 
responses has led to developuient of individual and proup intervievz check- 
lists and e>:p&rinentation vrith scalar designations of I'esponse (Lovjis, Llorris 
& Bendy 1971) as collected throufjh SUS procedures. A group iatervievT con- 
si?/ting of four itcn^s designed to detcrriiine v;hether or not a student liked 
the film is administered follov7in3 the. filua presentation and posttest. It 
v;as found that EIII children did not dCinonstrate the ability to deal vrith 
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alar semantic rGsponsc?.^ and that experimonLcrs vera unable to comunicatG 
the response procedure or criteria to the cirildren. Scalar rer:ponse re-- 
quironents T?era sicplifiad to three cate;3orieo of l/ilcini. Students are., 
asked whether they liked the f i.1.>:"i . '"not at all", ''a little'^ or ''a lot*. 
In another question^ ctuJcats are asked V7hether: they vould like to see the 
film again. Bond (197?.) found hijb. correlation (-95) bGtv;aen a student's 
response as ''liking a presentation'' and ''wanting to see it agalnV. 

Attendiyig Dehaoiox^ 

After a nuiuber of experimental trials, a technique of obtainius 
attention mi?5asures evolved by takin;^ observations of the ''eyes on the 
screen*' behavior of a group of children viev/ing a film presentation. The 
observation consists of recording the nuvnber of a selected sanple of 
children watching a visual prescnv.ation at specified intervals so that 
comparisons can be made later based on the resultir:;^ frequency curve for 
the visual experience pr'ecented* lV;o diTnensxons are considered and 
collected siwultaneously ; the number of subjects watching and interval 
of time. After soiae experimentation with length of tiiae intervals and ■ 
subject snii.iple S7..zes observedj, a Icive-second time interval and a sainple 
of five randomly selected students v/ere determined for one data-'gatherlng 
event. In investigating the question of inter--observer reliability. Bond 
(1971) found agreement between observers to be .95 and greater, leading 
to the decision that only one observer V7as necessary to record an attention 
profile for a given filiT) showing. 
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To ccv.pi.le attention profiltiis for a given filTiij composites ate 
tnnde of obscxved san'ple behavior* During a prcGentatioUi observer 
posH-J.ors hinGelf in such a way as to observe clearly the 5 oubjecLG* 
eyc^.s as th&y view the fili-i, recording attending behavior on. the atten- 
tion profile form. After several pronentations ^ the samplas ox 5 
subjects v7ho srvj the saute film are compiled into a time series line graph 
indicating the frequency of subjects v;atching (vertical value) at each 
five-second interval (horizontal value) (See Figure 2-4) . This cor.:ro3ite 
frequency graph becomes a visual representation of the attantive behavior 
e>:hibited during the length of the media presentation. Time Series 
analysis, particularly the non-stationary moving average model (Bo:^ £t 
Jenkins, 1970) is particularly V7ell rjuitcd for reducing attention obser- 
vations to sunrmary information and statistics for couiparisons betvTcen parts 
of a profile. 

20 

F 
r 



Q 


15 - 




vV 






JL 








5'; 






vV 






5'S 


V'f t': 






u 












V? 


^: 






vV 






sV 






^< 


'/< Vc 'A 






e 










^. 


vV 


-k 


:\: 


it 


'k 






vV :V 




■/f 


;'r 


5V ;V 


i: ^ 




n. 


10 - 


'Ji 






v'; 




i'c 






<V 


ii 








^•j 




V< v'? 






c 




V'; 






vV 




;V 










Vt 


5': :V 




5V 




A* vV V: 


VV 


y? 


y 




•/< ^ 






;V 












vc 


:*< 






i'c 




*\ t\ (C 


>V '/c 


•A 




5 - ^ 








vV 












V{ 






V* 


:^ 




•^V :V >V '/i 


v': 










*% 


.J. 


■A 


Vf 






*^ 








:V v'f 


>V 






•/r ;V ;V 
















;V 


:>< 






v'c 








X ic 








/c -A* 







.15 30 ii5 1 2 

Seconds 

ilinutcs of rrc.sontatrion 



Fiauro 2-/4: Grapliic Sol.id of Attoution Profile for 24 Children 



ERIC 



- 10 - 



The i;Ocond profile preyc-:nt:rl:ion :l.s a t:J.nc graph (Figure 2-5) rbowing 
plots of up to three neparcte clci.sGif icr/clonn (priiparyj intiern^cdiate , and 
secondary or 3 diiferent: obs.-'vvors) of the observed population slriH-il tan- 
eously froja the saniu rav/ data i.iatrix as the r>:olid. Curve snioothinc is 
dene by a r.ovins average procciDS, v?hich cveragOG the prcf^ent ojL'crvation 
with the o!ie prior to it and t^ie one follotTlng to produce a svntoothlng 
of the individual observatious . The rer>ultlng curve is plotted v?ith time 
as the K axis and frequericy of observations a3 the Y a>:is (Greenfields 
1973). 

Bond and Pruzck (1972) investigated the ii^;e of tiuie series analysis 
to compare attention profiles for significcint deviationn. The lj:.iitations 
of the conceptual franiei;ork of time r.ieries analysis and conputer prograins 
fcir time series analysis nake this procedure x.^orkable only when specific 
points in a given profile ha-^^c been specified. Th.is is done by first det- 
eriiiining visunlly the differences in the peaks and trouf,hs of the attention 
profile prepared in cither forva. The techniques to coi^^pare separate 
series are under devolopTnent and may be available in the future. At 
present J though, the ^^eyebail^^ of differences in portions of a profile 
or of indepcnoent profiles wjII have to suffice. 
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Chapter III 

InveGtijntions Concarnii^S Testing Effects Tvith 
Educable Hcntaliy Handicapped Children 

A major concern guiding the research investigations^ at the Computer 
Based' Project has been thatf of testing ef fects on the population used by 
the Project. Twa aspects of the general question are addressed in this 
chapter: the effects' of pretegUlng and the effects of response modality. 

Effects of Pretesting 
The possibility of cuing effects in the use of a pretest in niedia 
evaluation procedures has been of concern. Tlie Solouon Four Group design 
V7as conceptualized, but raodified to conserve effort and subjects by elimin- 
ating the pretest-only and posttest-only cells. A simple pretest-posttest 
design resulted, where the subjects are their ovm control* In a number of 
situations, hoxv^ever, administrative activity resulted in data sets that 
fit into the Solomon Four, and these were used to check pretest effects. 
The premise that pretests cued the subjectG was rejected (Bond, 1970; 1971). 
It was found, rather, that the use of the pretest tended to depress sub-- 
sequent posttest scores approximately 10%. The results were attributed to 
the procedural requirement of lengthened attention span and possibly lowered 
ii\terest level when a pretest vjas administered before the showing and a 
posttest after. Thus, x^osttest scores obtained from the media experience 
vjere lower because these items appeared at the end of an experience, and 
the likelihood of sustained viewer attention v?as less than if the 
lengthened expririence had not included a pretest. 

O 
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A sGcond observation X7as the tendeacy of r,ubject£^ not to answer 
an identical pretext item v^hsn it vras prcisentc?.d again in the posttest, 
p^irticulariy If the subject thought he had ansvrered correctly in th.e • 
pretest experience (Bondv 1972 b). 

The effects of tixe spacing for pretests prior to seeing a film 
were studied during the first year of film data collection. Little 
difference vc.s found between the pretest: or gain scores for tests given 
one nraek prior to the viev:ing of the f il»n v/hen co\Tipared with pretests 
given directly before a vievring. The pretest v/as administered in one 
of two situations: (1) after the shov7ing of an unrelated film and its 
posttest itcais'5 or (2) prior to shDV7ing an unrelated film* In the first 
case pretest scores were lower than in the latter case. Host of this 
difference could be attributed to the attention span of the subjects, as 
mentioned earlier t Administering the pretest items and ten posttest 
items after a film shoving made a rather long session. There seemed to 
be a tendency of the children to "hurry through" so that they could get 
on to the next more desirable exercise or activity. In the second casej. 
this tendency of the subjects co hurry through tlie unrelated pretest 
items so that they could get on vjith the more desirable activity of 
watching the film seemed to be present; however, the subjects v;erc fresh 
and v;ere observed to be more x;illin'S "put up with the test'' as a 
prerequisite to being shown a film, No evidence was observed of the 
reme:jibering of the pretest items from the v;eek before after seeing the 
film preseiitArlon and c^^dHnj^ the posutest. 
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*nie pretest-posttest desigu was varied in the thij.d year (1971-1972) 
of film data collection to allov; the use of ''prete:Jt-only*' groups as 
filiTi viG^jins groups. This uas done by i^rlr.ring tlie ccl::- of itcrrif: fro3 
tv7o films. The "seen*' film itans v;ere related to a film the group viex-jca; 
the ''unseen" items were related to a film they did not viev7. These 
mixed iteras were presented as a single test follovring the film showing, 
thus eliminating the lengthened presentation caused by a separate pre- 
test administration before the showing. The possibilities o£ cuing and 
repetition of the same item v;era not present in this latter arrangement 
because the item v/as seen by a different 5 independent samplec FolIov7ing 
the test, the items V7cre sorted into the two film groups (seen and unseen) 
prior to analysis. Findings indicated no change in level of pretest 
(the unseen film) scores as corapared V7ith previous saparate pretest 
administrations (Bond > 19 72a) • 

Response ModaVitieis 
A second concern has been the mode of collecting student responses. 
Difficulty had been experienced by s'pecial education teachers in getting 
appropriate 5 valid responses from Klftl children during group administrations 
of standardiJied tests, raising th.e question of whether or not these same 
children could be expected to give appropriate responses to items asked 
about films in the evaluation process • It v^as decided that an automated data 
collection system v?ould simplify question^answer procedures. The General 
Electric SRS 1000, an electronic system providing each respondent: V7ith 
five denk-top Tnounted buttons, interconnecting circuitry^ and a computerixed 
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analysis, has the capabjlity of providing ei'chcr iir^nied^iate or delayed 
couf Irmatioii of the correctness of a responsic. Several investlsations 
into these (Effects are revici;ed, 

EesuJts of investigations to detcrraine the reliability of SRS 
obtained responses with paper-pencil responses v:ere contradictory. One 
study indicated that posttost scores \7ere about 10 percenta;;^e points 
hi<^her V7hen the test V7as admiuii^tercd on the auvomated system (SRS) than 
\Aien done in a papers-pencil mode (Bond, 1970), In aiio^iier study^ however, 
a number of the SRS stations v/ere inoperable fur several data collecticn 
sessions- A paper ansv;er strip v^as f^iveii for those students at the non- 
functioning stations to record their "ansvycrs* Coi:iiparing the paper and 
pencil scores V7ith the SRS responses sho\:ed no si^inificaut differences 
betvreen SRS and ansv7er 5heet scores (Bond, 1971)* A similar finding 
resulted \jhen the posttest scores of otherv7ise similar EJ-lil groups > one 
using the SRS and one using paper strips 3 vyere compared* 

Some difficulty was experienced in the data from the primary level 
students usinr; the SRS systeta. Preliminary investigations indicated 
that nearly half of these children could not distinfiuish the numbers 1 
throujrjh 5 (Spaid, 1970). Several experiments with pictorial and color- 
coded response sheets had not significantly improved the correct responses 
of primary children (S;:)ith, 1970) « A training sequence for the SRS was 
designed to teach these children to match nuiabers and other answer choices 
V7ith the appropriate answer button (llorris^ 1972) with a resulting in- 
creor;e in dis:ernable result^i from a given question set in the SRS facility. 
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Flowers and Lond (1972) studied thci G^ffects of having the same 
response number for all items end revealed that interip.ediate E12i 
children began to learn (or catch on to) the appropriate response after 
six to nine iterasj V7hen the saiue nuiubcr-button vzas used as the correct 
response to all itews. 

A study by Anastasio (1972) revealed that the use ot ansv7er con- 
firmation on pretest responses had little training effect on cuing students 
as to the correct response for that same item, if it appeared on the 
posttest after viewin[^ the related film. Although the children seemed 
to be playing a game of ''getting the most correct responses*', it was 
done in more of a gambling spirit than one demonstrating prior learning. 
Ko differences vrere observed on posttest scores betueen confirmed and 
unconfiraed pretest scores. 
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Chapter IV 
Investigations Concerninc Vocabulary 

One variable v;hich may be important for learning by children 
frcH sound-narrated or captioned filViis is the vocabulary used in the 
narration or captions. Investigations have been conducted by the 
Project to identify and validate v:ith E'?.I and SSR children a 
vocabulary list that could be used to measure the vocabulary level 
of the captioi\£r or narrations on films. To date the investigations 
have been tx:o-partr, validating of applications of an available voc- 
abulary list and the developing of techniques to process the text of 
the narrations to compare the uords V7ith the vocabulary list. 

Validation of Vocabulary Lists 
To identify a standard from \?hich to measure vocabulary levels of 
mediaj a search of the literature on vocabulary uas conducted and the 
coinpiled list, A Functional Basic Uord List for Special Students ^ found. 
(Tudyrfian and Groelle, 1953). This list contains 2483 words arranged 
by grade-level difficulty that are coiisidered by the coiTipilers to be 
the most appropriate for the day to day needs of the special pupil 
lcarni::R to read. The Functional Basic Uord List ^ -is divided into three 
levels as folloxjs-: 

L&vcl One consists of 1,226 v?ords vjhich ''reflect the Interest 
and social living experiences and social maturity levels of the 
child.'* There, are five sub-levels for students v/ith a chronologi- 
cal age (CA) of 8-0 to 12-^6 j a social age (SA) of 7-0 to ll-o, and 
a mental age (llA) of 5-1 to 9-3. 
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1- A * 1? r e-pr iiTii^.r 



( 75 vjords) 



1-3. 



F rimer 



(142 words) 



1-C. 



First Grade 



(279 xjords) 



1-D. 



Second 



Grade 



(324 words) 



1-E, 



Third Grade 



(406 v7ords) 



LeVi^l Two consists of 641 v/ords suitable for students V7ith CA 12-7 
to 15-6, SA 11-7 to 14-6, and VA 9-0 to 10-3. The 616 v:ords in 
Level Three are suitable for students \7ith CA 15-7 to 18-0 j 
SA 14-7 to 17-0, and I3A 10-0 to 12-0, 
The list was fouiid' to contain taost words coiuition to other lists: Gates (1926), 
Thorndike and Lorge (1944), Russell (1948) ^ lietts (1949), lIcLatchy (1951), 
Rinsland (1953), and Dolch (1956). The list wes tested with Project popula- 
tions to verify its classifications and adapted as staiidnrd to measure 
vocabulary levels of media, 

Three studies V7ere directly concerned V7ith list validation. Lev;is 
(1972) established that the levels indicated by the list, particularly the 
subdivisions of Level Chie^xrere valid for the City School District KSB. 
children. 

Basic instrumeuts and procedures for testing vocabulary levels v;ith 
Biffl and SSll (emotionally handicapped) children were developed for both 
reading (v7ritten) and oral (aural) vocabulary to deternine if the subjects 
tested knev7 the v/ords that Tudyraan dnd Groelle suggest they should know. 
Three procedures v;ere developed based upon the following definitions: 

Bead: when a student is presented with a word in writing. 



he will respond by emitting the sounds associated with 



that v7ord. 
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Written meaning: vjhen presented with a v;ord in v?ritings 

the student V7ill use the V7ord in a sentence* 
Oral m-^aning: v/hen presented v.'ith a spoVcen vjord, a student 

v/ill j^espond by using the woird in a sentence. 
Criteria: The experimenter determines that the subjects used 

the V7ord in a (1) gra^Tnatically co?:rect Tuanner (i.e. a noun 
is vised as a noun) and (2) a conventional tnanner (i.e. ''I sav7 
the cat*% versus ate the cat") to demonstrate his capabili- 
ties to associate the vjord with its referent. 
The first procedure, a Verbal Recognition Tesr, measured tl).e subject's 
ability to read orally a ^»7ord presented frou the printed page. The second 
procadure, a Written Uord llcaning Test, determined x;7hether a subject knew 
the meaning of a printed wordo Subjects vjcre asked to use the word in a 
sentence. All students sair.pled were able to give a response to this request 
in such a way that the experimenter could iiiake a judgment based on the 
criteria listed above. The third procedure consisted of an Oral VIoi'cl Meaning 
Test 5 in x^^hich a xvrord v/as presented orally by the e>iperiinenter in addition 
to the printed V7ord being shown. Trials on ten EUl] children provided satis- 
factory results for dcternining subject levels on the list, 

Lewis concluded that the students tested could read the v7ords as recom- 
mended and v?erc able to give meanings of words at a level higher than ex-- 
pected V7hen the V7ords V7ere presented oral].y. 

To use the Fioictiondt Basic Word List for analysing and classifying 
film captions and audio narrationj, it was considered necessary that the 
Project invest io,ate \?ord-lcvel aasosr^ment for variant forns of list v/ordsr 
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specif ical.ly J regularly formed plurals c'.nd past tease verbs. To assess 
the level of plurals (Wood and Bondj 1972), a roviev? of the list itr;elf 
v?as first conducted to detcin-iine the treatirient of plurals. Four conditions 
of plural listing were found: (1) plurals without singular form in the 
list- (i.e, ashes); (2) plurals lacking a couimonly used singular form 
(i.e, clotheS:, dues^ politics:, slaaksj trousers)', (3) irregularly forniGd 
plurals for v^hich listing preceded the singular fonn (i.e. children^ 
fect^ leaves^ mice); (4) irregularly formed plurals that followed the 
singular forra listing at the same or higher level (i.e, ^7;^??;, wonien:^ 
cookies^ matches). In general, however, plurals formed regularly through 
addition of s or es were not included as separate entries. 

To deterrjine the level of regularly forniod plurals, izivG jcigations 
with EIM children v?ere conducted. Procedures included the VJritten Word 
ileaning Test and Oral \7ord Heanlng Test developed by Levis (1972) to 
validate list levels V7ith the Project population. Further procedures 
vjere developed to deteraune students* knov/ledge of' the plural, depending 
on the apparent ease vjxth v?hich subjects constructed the criterion sentences 
for the singular form. A subject unable to build sentences easily v/as 
supplied with frames calling for plurals. An additional procedure to 
test aural word recognition and discrimination betTjeen singular and 
plural fo?;ins was developed for students who appeared to have difficulty 
V7ith verbal e:rpression. Simple line drawings representing iteti^s cor- 
responding to the vioun labels v^e.re presented p and then a student was 
presented a singular or plural form of the V7ord orally and a^iked to 
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point to th?. correfjponding iriiage. Thus one could test whether or not 
the chilrl could associate the orcilly presanted noun with its ccrres- 
pondins iconic represents cion ar-d wjicther or not he discriranated 
betv;een lone singular forins and pltiral grouped figures. A variation 
of this procedure occurred as several students became enthusiastic 
and asked pamission to inake their ovju dra:vings. In this case, the 
examiner presented the singular form of the word orally {dog^ box^^ face) ^ 
asked the student to draw what he thoughc t.V:e v7ord represented » and then 
asked him to drav7 the plural forms ^ dogs^ i>oxeG/ 'fa,ceG. Several of 



the V7ords selected, especially na^es for parts of the body'^-;^-,|;,« tested 



Tlie findings from the invest?.gation supported the hypothesis that 
if a subject knows a singular word.^ he v/ill recognize the .? or es 
variant. The investigations exarained only base-unit recognition. No 
attempt vjas made to determine whether or not the students tested recognir^ed 
the concept of plurality. 
Past Tense Assessment 

The question of determining level of word-list variants found by 
adding past tense morphemes x^ras investigated (Kessler and Bond, 1973). 
Difficulty v/as encountered v/ith the sample population when trying to com- 
municate the concept of *'paGt*% "past tense'^, or "it happened yesterday'^ 
to the El'Til children. Contrary to Lewis's findings , nost of the priT7?ary 
EIH subjects were unable to read the stimulus v;ords (verbs) as required 
for the Verbal Recognition Test and Written IJord IJeaning Test. The Oral 
VJord i-eanlug Test v7as tried. Subjects could repeat the word orally but 




for aural recognition by si'a.ple pointing procedures* 
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had difficulty u:?in2 the v?03:d in a sonteace. Their efforts tended to 

result in a senrcace locking the stipulated word or a meaninf>less 

group of contrived A-7ords that vas uiiinterpretable. A third technique 

was developed usln^; suggestions made by vJood. A child vras engaged in 

a iTiore informal activity v/here he v;as C'L^en paper and crayon?5 and asked 

to create a picture of himself doing :;oi:iethin3 he liked to do. Unen 

finished, the subject vjas asked to describe sorrie of the thinr;s he had 

done the previous day. The iiittirviev7er n-ade mental note of th.o use of 

the present and past tense verbs in the deccription, Foll.owing the 

picture making and narrative interview, the exaainer displayed a card 

bearing the present form of a verb and read the word. The examiner then 

employed techniques used in the Listar Lan[7,uage Pror^ram (Science Research 

Associates 5 1969) where children are supplied \7±th a verbal unit and 

asked to 5.initate it as shov?n below. 

Exa^^ple! The student v;as osked, ''Did you danco yesterday?'' 

S *'Yes." (If a No ans-^er is 

received;, another verb is used-) 

E "Say the whole thing.'' 

S "^''esj 1 dance (d) yesterday." 

The expTniner noted the word form used in the second response. If th^ 

ed form was audible, the subject was scored as denionstrating knowledge 

of the pas:t tense. All the verbs on the list were subsequently substituted 

for tb.e underlined verb in the exoKiple above. 

It was observed that in the verbal descriptiOiis of the subjects* 

activities s the past tense \yar; ei-ployed soncwhat frequently end arbitrar-- 

ily. A disregard for ccnveutional past tense for:a- of words wr^s 



especially marked in the descriptions by black children. These same 
children seemed able to conceptualise the cleaning of past tense v;hen 
describing their activities yesterday , although the pronvmciation 
of the ed v/as not ali/ays audible to the experimenter. 

The conclusion that, for Project purposes of vocabulary level 
analysis, the past tense form of a verb can be classified at the same 
level as the present tense V7as based on data V7here subjects seemed 
to be able to use the present and past tense forrus of v7ord.s in level l-B 
and 1--C. Some difficulty of xxsiug the past tense of verbs in level 
1-A vjas identified; l-ovjever, all subjects v;ere functioning at a llA. of 
6.5 and v/ere able to dciaonstrate that they knew the neaning of list 
words throush level 1-Cc The specific verbs in level 1-A may tend to 
be more useful in their present tense and used less by primary children 
in the past tense. Investigations v:ith level 1-B and 1~C verbs suggested 
more equal use of present and past tense. Conclusion was reached that 
Hill children of primry level seem able to recognise the past tense 
form of verbs near the level at V7hich they recognise the present. 

Teohniques to Analyze flarrative Text 
Once the Funct-ionat Basic Word List had been validated as an 
appropriate standard for the E^IH populations developn:ental efforts were 
begun to create computer programs to refine the task of analyzinij texts. 
First efforts involved a staff BLernber^s transcribing* the caption or audio 
narration, tallying frequencies and tuaking comparison to the Funotional- 
Basio Word List to obtain level classification to verify that the v/ord 
was not in the list. Realizing that all of these tasks v^ere moro or 



leso mechan:lcal in natu}:a and involved Liialcing cor.parisons to determine 
frequencies 5 adding to tlie listr, alp^:^'.bcti^in<!; ^jnd chec>.in[^ for list 
inclusion J four coniputer progravy^ (UrJDLST) were written to handle the 
tasks (Greenfield, 1973). 

VIRDLST accepts the narration or text in sont-cncc form on cards 
or paper tapes and produces an output consistin,?} of au alphabetical 
list of all v7ords used iii the narrations the frequency of use, the 
Functi'rOnal Basic Word List level clasi^if ication (seven levels) for each 
word (or an for those not in the li5:it) and a summary of th^^ frequency 
and percent of vxords at each level and those not lif;ced. 
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rhapter V 

InvcstigationG Concernin^^ Children's P.oi^ponsos 

The Project model of madia evaliu' l'ion is learncr-bosed , requiiring 
that students as evaluators indicate by their behavioral responses 
that the materials being exait^iucd have or have not been effective. 
Such a nodel necessitates the gathering of inforru^cion about the Project 
efforts to iTveasure the cognitive and affective responses of Elfrl children 
5-n concert with, or as a rcGult of^ th^: viewing of instructional filiuS, 

A number of observed Ihr.itations have been recognized as character- 
istic in this test population of EiiH- children, as discussed previously • 
They are unable to read many commonly used words and limited in verbal 
descriptive expression. Bond (1972) reported the results of taped personal 
interviev7s v^ith EilH. children of primary;, intermediate, and secondar)'^ levels 
for the purpose of assessing; the effects of a specially selected fi].m, 

Very Special Day", and found conf irFiation of the verbal liBiitati ons of 
these students* Leading questions v^ere asked such as ''Who V7as in the 
film?^^ c;ud 'UTnat v;ere they doing?" in an effort to get the childieu to 
talk ab(?ut thej.r cyperiences- An analysis of the transcribed interviews 
revealed that: 

(a) The children gave short one-^vrord responses to the 
interview questions. 

(b) All levels (primary ^ intermediate and secondary) of JiIlH 
children exhj.bited similar responses . 
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(c) If a quCfjLj.on vrere stated calling for yes or no response > 
a student could identify settings, sequences of events and 
simple cause-effect relationships, Studcntis did i:iot vcrbcilise 
these events voluntarily, 

(d) Concepts such as f ather-daughterj boy-girl or peer relation*- 
ships 5 were essentially uon-e:2:istent in s;:udent responses, 
Anastasio (1973a) observed that chil.dren v?ho vrere being interviev/ed 

had difficulty Jascribing their erperience after vicv^ing a film and 
deiGonstraterj that those children V7ith low verbal abilities could in- 
dicate a concept learning by using a photo sequencing task. He 
reported that standardized IQ tests such as Stanford Binet used a 
sequencing task and that Frostig and IJaslov? (15 08) and Meeker (1969) 
had identified sequencing as a visual motor skill to be eipphasir.ed. 
Using these concepts as a basis 5 Anastasio set up a study in \7hich one 
group of children v;ere asked to sequence 5 photographs portraying a 
concept that SKoking has ill effects on the lungs. Another group of 
children v/ere shovni the film *^Bq Smart, Don't Sdart'' and, following 
film presentations asked to sequence the x^hotos and describe the mean-- 
Ing the sequence coimxiunicated. Those v;ho saw <:he film v/ere also 
interviewed > using a standard format. Scores v.^ere derived for the 
sequencing activitiy^ the verbal ^ visual, and auditory parts of the 
interviex;, and the judgment of the ervperinienter on concept definition, 
Con»paring tlie sequencing scores with observers' judgments of student 
verbal ability revealed a positive^ but not significant j correlation . 
of •A0^ The sequencing scores \':ere signi.f icaixtly different for those 
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seeing the film as compared with those uho had not, suggesting 
that the concept o£ the ill effects on the lungs of smoking can be 
measured using sequencing techniques and circuuvei-^.ting the verbal 
havidicap of Kill children. 

Of considerable importance to CBP procedures vzas the question of 
whether or not children's attention to a film influenced cosnitive test 
scox'es. The use of pretest-posttest questions to determine the instrpc- 
tional effect of a film has been sumrp.arized in Chapter I. Generally, 
the basis of selecting itciiT;3 for criterion instruments has consisted 
of adult's judging the pertinent content from previewing the film and 
studying prepared outlines, then creating items v/hich reflect mastery 
of this content. Alternant ives have been investigated, hov/ever, to 
determine other possibilities. From the attention profile of a given 
film the high points and lov; points vjere Identified uith the content 
concept being presented (Bond and Spaidj 1971) and items V7ritten speci- 
fically for these highs and lovrs. After shoving the film to other 
groups, the correct responses to these items V7cre grouped by high point 
and by low point. No discernible differences v-^ere found betv?een the 
number of correct responses on a subgroup of items V7ritten for the low 
points and those for the high points. The correlation bo.tv7cen the 
attention profile obtained in the latter case and the one used to vrite 
the items v;as ,79, indicating that they vjere not decidedly different. 
The profile for pr*imary children v/as similar to the profile for inter- 
mediate children^ 



- 40 ~ 



Obecrvration of ritudeut aUtending behavior during filra presentations 
raised further questions. A r.earch of the lite>:aturc on n^iasurement of 
attention was conducted by Lev;if? (1972a). He found a nuir.ber of suggestions 
for technical procedures vjhich eventually led to the construction of a 
special, facility for using conjugate rcinf orcenaiit techniques: a pro- 
cedure in vjhich the stiuiulus is also the reinforcement, Lev7is designed 
and built the facility as a tx/o-rcoin systeiu. Observation is conducted 
in one roora, the student responding in another. The capability of record- 
ing profiles of response frequence es^ audio responses and visual responses 
is liuide possible. The subject views the KiedJur.i on a rear projection screen, 
hears the audio through a speaker, ^md responds by depressing one or more 
sv/itches mounted in the arms of the chair. T\>7o or more slide projectors 
provide capabilities for simultaneous projections such as visuals and 
captions. A 16 mm projector provides inovia car-ability. 

Lev/is (1973) conducted a study using conjug^ite reinforcement proce- 
dure to measure attention to stjmvilus color (color or b].ack-and~uhite) 
and degree of real ism (picture or drawing) . In the lyTooaouve^ the 
presentation time of the stiviiulus varies directly and iniaediately V7ith 
the subject's rate of re^^ponse. In this particular study, the screen was 
lit V7ith a stimulus for one second each time tlic subject emitted a 
response. Results indicated that none of the independent variables 9 (color 
black- aad"v;hite; picture, dravring) had significant effect on the atten- 
tion me^HGure. IIovTever; a trials by color interaction and a trials by 
color by sex by realisia interaction v/as observed. In the color by trials 
interactions the black-and white group shov/ed higher aLtention scores 
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over triali; vv-hiJe the color sroup showad lovrer scores. Posttest 
data shci'jed that pictures v^ere sic^if IcaLitly preferred to drir.nnRS 
by aJ.l ^^roups. Other cuestions currently being investigated through 
the facility include the use of captions. 
Affective Responses 

The Project recognized early that films had more effect upon the 
vievjer than could be measured frcm a fev: cognitive items presented after 
the film shovjing. Early efforts to collect this iuforniation from inter- 
views proved fruitful, and relevant to the evaluation efforts. Krathwohl 
BLooin cud ilasia (1964) categorized the affective domain as cones is ting of 
a continuum of five major areas; rcceivingj respondingj. valiixng, organi 
eatioii, and value characterization. The first three are divided into 
three sub-areas; the lajt tvjo-, into tJ.:o sub-areas. The attention profil 
data cojlected by CBP is obtained by recording sustained eye contact of 
the viexzer \7ith the presented visual: it is categori^icd as receiving - 
selected attentton. The liking response in the interview it:cii:3 is 
categorized as valuing. The relationslilp of data collected In the evalu 
ation process and the affective domain is rather loosely conceived; 
ho^/ever; by ideiitifying efforts v/ith an affective domain categorization , 
it is hoped that the results indicated by the data can ha explained in 
Tcore gcneralijsable terms. 

Smith (1970) noted that secondary cliildren began to read aloud the 
captions on filmatrips as she presented the visual and read the caption. 
Intervie\?s vyith the student revealed he had 'discovered he could read."^ 



- 42 - 



Thio e>:c:Lt.a.ic:VLl: r(as\i.\tc:i in an obs:i-?rved interosfc to V7ant to partici^ote 
In future- sessions Indicated by a vocue.st of ^'IJhcn can I do this asain?*' 
erc,> or a plea to choose, bin ajaiu. Encoura2c-,:.C:at was given to have the 
siiiKlents read the captions t^ith ai^iiis Lance froia the observer v/heuever 
possible. 

Numerous observations and su:mnaticns by SRS operators suggested 
that some films seei;:cd to affect strongly the attitude of the viewers. 
Bond and y.osin^ (1973) sot up a derrirable attitudinal mcdcl consisting 
of negative responses to ov;i^in3 a gun^ hilling auinals^ and huntings 
using statements alluding to these topics and ref»uiring the viev7er to 
indicate his position in a dichotcraous or trichotojious scale. Viewers 
v/ere pretested, shovm the film ''The Hunter '\ mid posttestcd. The 
results defeated adult conceived attitude models. Tae rer:ipondent3 
moved further from the proposed model as a result of having seen the 
filxap rather than toward the raodel. 

Investigating children's judgramts as to the age of a filmj 
/inastasio (1971b) found that their Judgments contradicted fact and 
teacher recoKiaendations • He showed filins which used fashions of 1950 
and 1960 to the children a^nd asked them to judge these against the 
question p '^Jas this film uiade .long ago or just iruide?" Teachers were 
a^ked to judge the films as being dated and usable or so outdated that 
they v/ere not suitable for instruction. Teachers rated the filras as 
being outdated and having very liirdLed academic value. The subtle cues 
of (lair, dress, and autoit^obiles were not mentioned and seoLned to be over- 
looked by the chx.Tdrenp however- Annstasio concluded that such cues 
do not necessarll}^ obstruct the content message in its effectiveness 
vjith K'iil chi?L.dren. 
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Chapter VI 

Investigations Concernluf^ Teaclicr Reaction and Opinion 

CBP staff rGCognisecl early the iraportance of finding out what the 
teacher had to say about media and its use, Throur^i vjorkshopa designed 
to provide training and guidance in the uae of ijiedia for special educa- 
tion teachers, throu[^h surveyo of teacher attitudes to\?ard the. work of 
the Projects and through parf:icipant observation, teacher attitudes were 
surveyed • 

Workshopo 

lleabure of teacher attitudes v;as fii'st obtained by CBP through a 
pretest administered during a vrorkshop for media vr>o held for 37 teachers 
in August 1971. Teachers indicated through their responses that they felt 
they could plan effective instruction for special education students and 
that they were generally not bothered by machines or computers. They 
indicated y ^ .itive attitude tcvjard filnf), perceiving special education 
students as learning control from f ilias • Posttest T^easureTi^onts at the end 
of the v7orkshop indicated inr:rease in positrLve attitude tov/ard films. 
Teachers also reported a strong desire to previcv; film materials and to 
use sorae standardlzad reporting or rating system consisting of summaries 
based on actual use of materials. Furthermore, teachers at the workshop 
indicated strong interest in planning for individualized teaching vn.th 
special education students. 

Efforts to determine more prec?lsely the parameters of individual-- 
i2:ation v/ere piloted by studying Structure of the Intellect (SOI) profiles 
using individual intelligence test (Btanf ord-Blnet and HISC) items. 



accordiiii^ to Heeker classification (1969) • Blasey and Head (1972) 
reported the findings of lirying to prescribe instructional activities 
based on the SOI profile for a given child. Tliese efforts were posi- 
tive and provided infonnation for additional training for teachers in 
using the SOI profile to organize EHH instruction. A unit consisting 
of six' hoiirs of instruction for teachers was included for the S workshop 
pavtici-jauts in Avxsust 1972> (CBP, 1S72) . These teachers responded by 
requesting computerized SOI profiles (Greenfield, 1972) for children 
in their classroom. 
Attitude SiiTOeys 

To those teachers cooperating with CBP evaluation procedures, 
Project staff provided advice and materials for enriching the in- 
struction of EMH children. The. form and degree of assistance varied 
from casual interaction and resource service to direct assistance 
V7i thill the classroom over a prolonged period of time. Observations 
of the effects of this personal service were recorded in both formal 
and informal ways. One could report to a staff meiiiber or in general 
meeting the results of a particular interaction request > service ^ or 
perception. Formal observation techniques consisted of surveys and 
participant observation. 

Johnson (1971) surveyed teacher opinion concerning Project sen/ices. 
T^^enty Thurstone-type iten-s were used to survey four areas: questioning 
and techniqvie^^ films ^ staff personnel > and general feelings. Johnson 
found rather neutral to negative response on most items? that the films 
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"though sortie times useful did not fit into any current lesson plan; 
that weekly visits V7ere disruptive of the class routine; and that 
teachers were generally not .kept informed concerning student progress 
or response. Teachers reported 5 ho^7ever, that the children seemed to 
look forward to the weekly visits, that they related well to the staff 
members, and that they desired to participate the follo\^ing year. 

'iiewis (1972) followed about six months later with an interview 
survey of 23 of Uui 28 teachers participating in the Project evalua- 
tion efforts. The survey investigated teacher attitude, teacher 
remarks that may affect stucJent p?.rf oriiiance, find knovTledge of Project 
operations. During the interviev; the teacher v/as rated as being 
posit ive> neuti^alj or negative on each of 30 ite^cs in three classif id- 
eations of students, teacher, and Project. Si::teon specific findings 

were listed and generalized into three general statements? 

i ' . , -■ ■ " ' ' . 

Teachers desire to be informed of Project activity. 

2. Films and criterion iteiES should be chosen carefully 
for level and class. 

3. Teacher opinion should be sought by Project persomiel. • 

Pavt^civmit Observation 

During the fourth year of CBP operations 3 a research team was 
organized to make extensive observations of 8 designated classrooms 
using participant observation (?0) to answer questions concerned with 
the use of media in the classroom and teachers ' use of- CBP services. 
The technique has been described by Eogdan (1972) <, 



A report sun'raarizing the activities of the first tv7C months 
(Bogdan, Brogcli-nj Dodge and Lev;is, 19?*.) had the following f^cnerjl- 
i'/ at ions : 

1- The ixic tructional value of a r.ediun does ao:: iiLiccoSai.lly lie 
in the r.ediuni but in hov? it is perceived a.-\d acted upon by 
those v7ho might use it and those v7ho are objects of its use, 

2. Improved teacher ef icctivenoss of necia involves getting 
teachers to use uew insights into objects cround them rather 
than providing more objects. 

3. How liiedia is defined by those in a classroor;: setting; is ir.ore 
important than the natvire of the media, 

IVo reports (Bogdan, et al, 1973 a^b) analyzed the over 600 pages 
of observer notes concerning the vise of media in the classroom* Tv/c 
categorizations v/ere made in the first paper for media use - group use 
and individual use^ Media was chosen for gioup if it vjas thought to aid 
learniup; related to readings writing, and arithmetic, if procedures for 
its use vrore made specific ^ and if it could aid die tcachr-^r in structuring 
tlia school dayv An additional criterion wa::-i whether or not a [^iveu 
naterial had been douionstrated as v/orkable by other teachers « Iledia tr^iide 
iiot to be :')itegrated into the curriculum but used^ rather to fill gaps 
and to entertain o Films ^ particularly 5 px*ovide a breadth 5 a coiarncn exper- 
ience^ and bri|;',btcn the classroom atmosphere. Teacl-ers tended not to see 
the film in ter^ViS of its capacity to teach co^^nitively and did not select 
them for this purpose. This finding seemed to be a variance vizh the 
generalir^.atlon above if one cor-ceivet: of the three R's as involving a 
j^roat deal of eognirion. On the other hn:'.idj the study fourid teachers 
not taking advantage of the affective potential of film.. These findings 
seemed to reflect a belief among teachers v:hc vic^.; mass produced media as 
not tailored to their needs or to student needs. 



Under the individual uses of nv's'lic^ Gome •j:a::hs<^.r startling obser- 
vations V7ere recorded. Teach&rs Td£iit actually refuse to relinquish 
control of the equipneat or nedia to a child, They saw most devices 
and materials us being useful for orJ-y one or tv:o children ^ hovrjiver, 
they v?ere quick to point out that designers need to have definite ideas 
of the type of student the materials are prepared for^ Tcnchers 
appeared to see theoe materials as adjuncts to their primary resources, 
as onrichvoent to be ijsed in free time periods > or as rev7ard:-j for good 
behavior. The limiting conditions of a typical classroon make student 
use of individual devices cind materials difficult because of the ^'gerried- 
up'' facility to provide for it. There was sone evidence that individual 
devices and materials tended to liiirit socialisii^g and break up teacher 
control techniques- 

In the second media-use report , -he authors 3urr.i:iarised sorae general 
trends in terms of in}\ox)at'lo'>i-dTOi)'-ofj^ effect^:, noveliy effeats^ squirrel 
offectS:, and ho.ngupo and liazox^de of media use. Of iivi^xvrtaivce in the 
document is a theiiie suggestinf^ that in ^^pecial education classrooDis in 
the city schools the written word is all important and the triangle of 
teacher-written v/ord-student is the domain where academic learning takes 
place- 

The innovation- dz^op'-aff effect v;a^ simply a decrease in use over 
tiiiie. Teachers explained tlLN?r::e effects in tenns of a raaterial^s bein^ 
percelvfr?.d as inappropriate to the ueL-.ds of the teacher and the needs of 
the student* iiedia novC':7Ay has potential affective educational va3.ue by 
arousing interest? providing enjoyment- and breaking up the routine. 



7.'e:0cher3 see.iij-:::d to actively exploit the novelty effects of mediae many 
times by default rather than planning. The equirrel effect V7as defined 
ac tha situation Xvhare a tcr.chcr checks cut cquipr.icnt and raatcrials and 
hides them in the room so that no one else has access to them. The 
hangups and har.avds of media use summarized observations that the lov; 
use rate of media, other than books > chalkboards and teachers, is due 
to the inroriVenicncG as.spc^ated vzith the niediaj related equipment, envi-* 
ronmont^ control and electrical support. The common fear of shock 
associ.ated v;ith electrical appliances carries over directly to any devices 
that use standard electrical outlets. Many classrooms vzere found to be 
inadequately v/ired for the demand a class using individual devices may 
make on the system. The delays and incompetencies experienced in the 
irepair of broken equipment V7as the m^ost common incoavenience. All teachers 
were found lacking training in alleviating simple problems such as a burned 
out bulb or a jammed fij.mstrip. 

The participant obser" ation studies refuted several findings reported 
frcm the teacher interviews and surveys* In the surveys teachers indi- 
cated a V7illinj?,nc5s to participate in Project activities 5 yfX PO reports 
confirmed that media vjere not worked into curriculum and teachers did not 
use CliP personnel. Teachers indicated a need for materials in cognitive 
areas, yet PO findings indicated that materials vjere not used to develop 
cognitive skills. During the v^orkshops, teachers vjere found to be oriented 
toward individual instruction yet PO revealed the classroom as group 
oriented and media considered as a group activity. 
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